The present invention relates to coverage tools which are used for both verification in the execution of programs and verification in the operation of a hardware system, and relates particularly to a coverage measurement tool enabling a user to create a solution for coverage models of his choice.
Verification is becoming one of the biggest problems of the software industry. The cost of verification as part of the development process is usually between 40-80%. The practice of letting the users find the bugs and fixing them is in the next release is becoming dangerous and costly for three main reasons. The first one is that the users expect the program to work correctly and if it does not, then, there is a cost to the reputation. The second one which is the actual cost of fixing the software could be very large when there is a large install base. The third one is that litigation can be used if the software error caused harm to the user. In software development, it will soon be required of a development to demonstrate that his product was tested adequately.
Functional verification comprises a large portion of the effort in designing a processor. The investment in expert time and computer resources is huge, and so is the cost of delivering faulty products. In current industrial practice, most of the verification is done by generating a massive amount of tests by random test generators. The use of advanced random test generators can increase the quality of the generated tests, but it cannot detect cases in which some areas of the design are not being tested, while other parts are tested repeatedly.
The main technique for checking and showing that the testing has been thorough is called test coverage analysis. Simply stated, the idea is to create, in some systematic fashion, a large and comprehensive list of tasks and check that in the testing phase, each task was covered. Coverage can help in monitoring the quality of testing, and assist in directing the test generators to create tests that hit areas that have not been tested before.
Many coverage tools exist in the market. Most coverage tools are program-based and check code coverage of the execution of programs. Examples include code coverage in C or VHDL and covering states and transitions of state machines. The main disadvantage of code coverage is that it does not  less than  less than understand greater than  greater than  the application domain. Therefore, it is very hard to tune the tools to areas which the user thinks are of significance. To overcome this problem, some domain dependent tools have been developed which measure specific functional events. In all these tools the coverage models are hard-coded.
Therefore, in order to make the tools applicable to many users, the models that are implemented in these tools are generic. This leaves a user that wants to cover models that are specific to his design with two options: to measure coverage manually, or to build his own coverage tool for these models. Both options require a large effort and are error prone. Therefore, specific models are rarely used.
Accordingly, the main object of the invention is to propose a new methodology for coverage of a design calling for separation of the coverage model from the coverage measurement tool.
Another object of the invention is to provide a single general purpose tool meeting all the coverage needs of a user both for generic and specific models.
Another object of the invention is to generate and implement a single general purpose tool which is oblivious to the coverage model and provide all the functionality of existing tools.
Yet another object of the invention is to provide a coverage measurement tool enabling a user to define its own coverage model, gather and process traces to measure the coverage of this model and generate reports on the results of the measurement.
The invention relates therefore to a coverage measurement tool enabling a user to create a specific coverage tool for a coverage model consisting of a set of coverage tasks specified by the user for checking a design such as a hardware or software system, the tool being associated with a database and with model definition means providing a model definition which defines the coverage model. Such a tool comprises an insertion engine for storing into the database a table containing the traces resulting from a plurality of tests, a processing engine for processing the traces stored in the database according to the model definition, and a coverage analyzer for analyzing the measurement results from the processing engine and preparing coverage analysis reports according to the model definition.
According to another aspect, the invention relates also to a method of generating a coverage measurement tool enabling a user to define its own coverage model consisting of a set of coverage tasks meeting certain conditions on the events that create the tasks for checking a design such as a hardware or software system, and comprising the steps of collecting event data from tests performed on the design and inserting the event data into an event trace, extracting from the event trace the tasks meeting the conditions in order to produce a trace task list and determining whether each task of this list is a legal task in order to update a list of the legal tasks when the task is a legal task or to put it into an illegal list when the task is an illegal task.
The advantages of such a coverage measurement tool are enormous. First, it allows the users to define their own models, according to their specific needs, and still enjoy all the functionality of a dedicated coverage tool without the need to develop such a tool. Therefore, it provides users with the means to measure coverage on models which were not available to them before. This tool also allows organizations to use a single tool for most of their coverage needs, and not be forced to buy and maintain a set of tools, one for each coverage model. Another advantage of using an external model definition is that it enables sharing of coverage models between projects. Finally, this tool enables the users to adapt their coverage models to their testing resources, and refine these models during testing. For example, users that can afford only quick and dirty testing can define coarse grain models with a small amount of tasks, while users that want to do comprehensive testing can define finer grain models.